.TH E1432_MULTIMAIN 5 E1432
.SH NAME
e1432_multimain \- Multiple mainframe information
.IX e1432_multimain(5) 5
.SH DESCRIPTION

E1482B MXI cards can be used to connect multiple VXI mainframes
together, so that modules in both mainframes appear to be in one large
logical mainframe.  This can be done either with an embedded
controller (such as a V/743), or with a MXI cable connected to a host
computer.

However, this multi-mainframe setup can be complicated, and it places
some restrictions on what measurements can be performed.  Before
delving into multi-mainframe measurements, you should consider whether
it would be sufficient to simply run separate measurements in two
separate mainframes.

There are two separate parts to getting a multi-mainframe measurement
working with E143x modules.  The first part is getting the VXI/MXI
configuration into a valid state, which involves setting lots of
switches on the MXI cards, and getting the logical addresses of all
the modules into the correct ranges.  Getting this correct is mostly
independent of the E143x cards, other than just ensuring that the
E143x cards have the appropriate logical addresses for the mainframe
that they are in.

The second part is getting the E143x modules set up correctly.  Mostly
this involves writing a multi-mainframe-aware host application, but it
also involves several SICL configuration files.

.SH "VXI/MXI Configuration for Multi-Mainframe Measurements"

Before even worrying about E143x configuration and setup, the VXI/MXI
system must be configured properly.

.ne 4
.B E1482B Revisions

Very old E1482B MXI cards have a problem with multi-mainframe
configurations.  The MXI card in the root mainframe must be revision
G2 or later, and the other MXI cards must be revision G or later.  The
revision can be found on a metallic label on the ``bottom'' side of the
MXI card.  If your MXI cards are too old, you may get occasional bus
errors in your application.  A bus error will typically cause the
application to crash, but will not crash the operating system of the
host computer.  As the number of E143x modules gets larger, your
chances for bus errors increase because more data transfers will take
place during a measurement.

.ne 4
.B E1482B Switch Settings

There are a large number of jumpers and switches on an E1482B MXI
card.  It is important get these all set correctly.  Incorrect switch
settings can cause anything from the MXI card failing to respond, to
bus errors in an application, to module hangs when talking to modules
in that mainframe.  I believe it is even possible to damage the MXI
card or the VXI mainframe backplane if the S1 and S8 switches (which
control VXI Slot 0 operation) are set incorrectly.

The MXI card switch settings are well documented in the E1482B manual.
This manual has nice pictures of what the switch settings should be,
both for the ``root'' mainframe and the ``non-root'' mainframes, and for
either an embedded controller or a host computer connected by MXI.

The next few paragraphs attempts to document most of these switch
settings, but it's generally a lot easier and less error-prone to use
the pictures in the E1482B manual.

The MXI card is the slot 0 controller for all non-root mainframes.
The MXI card is also the slot 0 controller for the root mainframe if
an external computer is used.  The default shipped configuration for
the MXI card is mostly correct for these mainframes where the MXI card
is the slot 0 controller.  There are two things that must change:

.ne 2
.nf
    * the logical address (see separate section below)
    * possibly the VME and INTX terminators
.fi

The six VME and four INTX terminators need to be removed for MXI
modules that are not at the ends of the daisy chain.  For systems that
use an embedded controller, this would be needed only when there are
three or more mainframes (with two mainframes, both MXI cards are at
the ends of the daisy chain).  For systems that use MXI to connect to
an external controller, the "daisy chain" includes the external
controller itself, so this is an issue when there are two or more
mainframes.

If an embedded controller is used in the root mainframe, then the MXI
module is \fBnot\fR the slot 0 controller for the root mainframe.
This requires several switch settings to be changed.  Table 2-1
``Configuration Settings'' in the E1482B manual lists the changes from the
default settings:

.ne 7
.nf
    * the logical address (see separate section below)
    * not the slot-0 controller (S1, S8)
    * MXI System Controller enabled (S4)
    * set the VME timeout to 200 usec (W6)
    * set VME BTO chain position to 1 extender, non-slot0 (W7)
    * set MXI bus timeout to 100 usec (W8)
    * do not source CLK10 (W9, W10)
.fi

.ne 4
.B Logical Addresses

The logical address windows required by MXI are explained in the
E1482B manual on pages 2-34 through 2-36.  The next few paragraphs
attempt to summarize those rules, but you may want to refer to the
E1482B manual as well.  The rules apply to all modules in a mainframe.

Each mainframe gets a window of logical addresses.  No windows can
overlap.  The window size (number of possible logical addresses in a
mainframe) must be a power of two.  The starting address of the window
must be zero, or an address which is a multiple of the window size.

Logical address zero is taken by the system controller, which is
either an embedded controller or a host computer connected by MXI
cable.  If it is a host computer connected by a MXI cable, then the
logical address windows of all mainframes must not include logical
address zero (since zero is in the host computer, not in any
mainframe).

For very-large-channel-count systems, it is best to allocate 32
logical addresses to each mainframe.  The first mainframe would have
logical addresses 0-31 (or 16-31 if there is an external computer at
logical address 0).  The second mainframe would have logical addresses
32-63, the third would have 64-95, the fourth 96-127, and so on.  This
ensures that each mainframe has enough logical addresses, and that the
system will not run out of logical addresses too quickly.

The logical address of the MXI card itself need not be within the
block of 32 addresses assigned to a mainframe.  One good method is to
put the first MXI card a logical address 1, the next MXI card at
logical address 2, and so on.

.ne 4
.B Dynamic Configuration of Logical Addresses

The MXI cards do not support dynamic configuration, and so must be set
to fixed logical addresses.  Typically the addresses chosen would be 1
for the first MXI card, 2 for the second, and so on.

The E1432 card does support dynamic configuration.  Dynamic
configuration means that you put the E1432 cards all at logical
address 255, and the system resource manager automatically figures out
an appropriate logical address and tells the module to use that.

For small systems, dynamic configuration is probably not very useful.
For large systems, dynamic configuration is quite useful.

There are limits to dynamic configuration.  Typically, dynamic
configuration does not work on the first card in a mainframe, but
works on all the other cards.  The first card must be set to a fixed
appropriate logical address for that mainframe.  The other cards in
the mainframe will typically get assigned logical addresses increasing
from the address of the first card.

.ne 4
.B E1482B Connections

If more than two mainframes are needed, daisy-chain them all.  The
first mainframe is the root mainframe.  Each mainframe after the first
is a non-root mainframe.

The MXI cables themselves are directional.  One end of the cable is
enclosed and has only single connectors, while the other end of the
cable is open and has dual connectors.  The enclosed end must be
nearest to the root mainframe.

The E1482B manual mentions a ``Level 2 Extender'' configuration when
using four or more mainframes.  Do not use that setup.  Use a simple
daisy-chain setup.

.ne 4
.B Embedded Controller VME Bus Timeout

This applies only to systems using embedded controllers.  Certain
embedded controllers try to detect VME bus timeout, and this conflicts
with the MXI cards which also try to detect VME bus timeout.  These
embedded controllers must be disabled from detecting the VME bus
timeout, typically by adjusting a jumper or switch on the back of the
module.

This is an issue for the E1405B Command Module, the E1406A Command
Module, the E1499 V/382, and the RADI-EPC7 embedded controllers.  The
E1482B manual shows how to adjust these modules to not detect VME bus
timeout.  However, note that the E1405B and E1499 are not supported
with the E143x module, and the E1406B and RADI-EPS7 are not
recommended by HP.

This is not an issue for the V/743 embedded controller, because the
V/743 VME bus timeout is much longer than the timeout used by the MXI
card.

.ne 4
.B SICL Versions

SICL is an I/O library used on HP-UX.  This low-level I/O library is
used by the E143x host interface library when communicating with the
VXI system.

Systems that use the E143x Plug&Play libraries do not directly use
SICL, and instead use the VISA library defined by the Plug&Play
standard.  However, on HP-UX the VISA library internally uses SICL, so
in the end SICL is used on all HP-UX systems.

Multi-mainframe systems generally involve a larger number of E143x
modules than single-mainframe systems.  This larger number of modules
has a tendency to find problems in the SICL memory mapping code that
runs in the HP-UX machine.  It is important to use the latest version
of SICL, which is somewhat less error-prone.  As of July 1997, the
latest officially released versions are C_03_09 for HP-UX 9.05, and
E.01.01 for HP-UX 10.20.  E.01.01 on 10.20 seems to be more reliable
than C_03_09 on 9.05.  There is also a pre-release unsupported E.01.04
version of SICL for 10.20 available at
\fCftp://hpls01.lsid.hp.com/dsp/temp/libsicl.sl\fR.

.ne 4
.B VXI and SICL Limits

The A24 memory space holds a total of 16 MBytes of memory, though
sometimes half of that memory space is wasted due to limits on what
the E1482B MXI card can map into the host computer.

The E143x modules each use 256 KBytes of A24 memory space.  Assuming
16 MBytes of A24 memory is usable, that limits the number of E143x
modules to \fB64\fR.  E1432 modules with serial number prefixes below
3647 use 1 MByte of A24 space instead of 256 KBytes, so only 16, and
sometimes only 8, of these older modules would work.  These older
E1432 modules can be upgraded in the field by HP to use only 256
KBytes of A24 space.

The limit of 64 E143x modules (including E1434) is an absolute maximum
for a VXI system.  The limit will be smaller if A24 space is used by
other modules.  Other modules that use A24 space include the E1562
SCSI Disk module and the E1485 Signal Processor module.

The E1482B MXI card allows a daisychain of at most eight devices.  If
an external computer is used, it counts as one device, so at most
seven VXI mainframes can be used in this case.  Also, the total cable
length of a MXI daisychain can be at most 20 meters.

For systems that use a very large number of E143x modules, the E1482B
MXI card can place additional limitations on how many E143x modules
can be present in each mainframe.  The E1482B MXI card limits the
amount of A24 memory used by each mainframe to a power of two bytes.
This typically means that you must use exactly a total of eight E143x
(or E1562) modules in each mainframe, or there will be additional A24
memory allocated to the mainframe that is wasted.  This waste is not a
problem unless you are building a system that needs most of the total
A24 memory space.

For versions X.03.15 and earlier of the non-Plug&Play E143x host
interface library, there was an additional restriction.  When using
MXI to connect to an \fIexternal computer\fR, the the maximum number
of E143x modules was only 8.  This was due to limitations in the SICL
I/O library, which have been worked around in versions X.03.16 and
later.

For users of the Plug&Play E143x library, using MXI to connect to an
external computer, the number of E143x modules seems to be limited to
8.  The limit can be increased to around 13 E143x modules, but only
when using at least pre-release version E.01.04 of SICL and VISA on
HP-UX 10.20.

As of August 1997, this E.01.04 version of SICL and VISA is not
officially released, but a patched version is available at
\fCftp://hpls01.lsid.hp.com/dsp/temp/libsicl.sl\fR and
\fCftp://hpls01.lsid.hp.com/dsp/temp/libvisa.sl\fR.  With these
patched versions of SICL and VISA, the limit is around 13 E143x
modules.

.B "Convincing SICL to use all of A24 Space"

The default configuration of SICL on HP-UX will not even try to use
all of A24 space.  Because of this, the maximum number of E143x
modules in a system can be severely restricted unless the
configuration is changed.

When using MXI to connect to an external computer, this limitation
will prevent using more than 24 E143x modules.  If any E1432 modules
are old enough that they use 1MB of A24 space, then this limitation
will prevent the use of more than 4 E1432 modules.

To fix this, and convince SICL to use all of A24 space, two lines in
the iproc.cf configuration file must be modified.  On HP-UX 9.05, this
file is at /usr/pil/etc/iproc.cf.  On HP-UX 10.20, this file is at
/etc/opt/sicl/iproc.cf.

.ne 16
.nf
Change:

	boot ivxirm -p -I vxi

to:

	boot ivxirm -p -M -I vxi

and change:

	sysreset vxi ivxirm -t &

to:

	sysreset vxi ivxirm -M -t &
.fi

After making this change, the iproc daemon must be killed and
restarted.

.ne 12
.SH "E143x Configuration for Multi-Mainframe Measurements"

Once the VXI/MXI system is properly configured, E143x modules must be
configured and set up correctly.

.ne 4
.B Old E143x Modules

Older E1432 (but not E1433 or E1434) modules have a bug in the VXI
interface which shows itself in one specific configuration.  The
configuration with the problem is a multi-mainframe system where an
embedded V/743 controls the first mainframe, and MXI cables connect
the first mainframe to subsequent mainframes.  Data read from E1432
modules in the non-root mainframes is occasionally corrupted.

HP LSD has a new VXI interface ROM that can be installed in old E1432
modules which fixes this problem.

In addition, older E1432, E1433, and E1434 modules have a different
bug in the VXI interface which can cause problems with the G5 version
of the E1482B MXI card.  This bug is generally quite rare, but can
cause "Data Handshake Timeout" or errors due to incorrect data
transfer to the host computer.

HP LSD has a new VXI interface ROM that can be installed in older
E143x modules which fixes this problem.

.ne 4
.B VXI TTLTRG direction

The VXI bus provides eight TTLTRG lines, which are open collector
lines connected to all modules in a mainframe.  The E143x modules use
two of these eight lines, one for a common sample clock, and the other
for a common sync and trigger line.  This allows multiple E143x
modules to sample simultaneously, and allow trigger events in one
E143x module to get propagated to all modules.

The E1482B MXI cards connect the VXI TTLTRG lines between VXI
mainframes in only one direction at a time.  Either direction is
supported by MXI, and either direction can be made to work with the
E143x modules.  However, we only document one direction, and we
encourage everyone to use this one direction, in order to make things
less confusing and to simplify support of multi-mainframe systems.

We always document that the root mainframe must generate the TTLTRG
lines, and all non-root mainframes must therefore receive the TTLTRG
lines.

Because the TTLTRG lines are driven in only one direction, there are
limits on what kinds of measurements can be done with a
multi-mainframe system.

We refer to the mainframe generating TTLTRG as the ``master'' mainframe,
and all other mainframes as ``slave'' mainframes.  If you follow our
default of having the root mainframe generate TTLTRG, then the root
mainframe is the same as the master mainframe.

.ne 4
.B Controlling TTLTRG Direction

The direction of the TTLTRG lines is controlled by two SICL
configuration files.  On HP-UX 9.05, these files are
\fC/usr/pil/etc/vxi16/ttltrig.cf\fR and
\fC/usr/pil/etc/vxi16/oride.cf\fR.  On HP-UX 10.20, the files are
\fC/etc/opt/sicl/vxi16/oride.cf\fR and
\fC/etc/opt/sicl/vxi16/ttltrig.cf\fR.

If the \fCoride.cf\fR file is used, it overrides the settings in
\fCttltrig.cf\fR.  Because the settings in \fCoride.cf\fR are less
supported and harder to explain, we recommend that \fCoride.cf\fR
\fBnot\fR be used for setting the TTLTRG direction.

Instead, use \fCttltrig.cf\fR to set the TTLTRG direction.  This file
is a little easier to understand - you specify a logical address for
each TTLTRG line, and the mainframe that contains that logical address
will be the one that generates TTLTRG.

The default for \fCttltrig.cf\fR is that logical address 0 generates
TTLTRG.  For embedded controller systems this works without
modification, because the embedded controller is in the root mainframe
and we want the root mainframe to generate the TTLTRG lines.  For
systems with an external controller, the logical addresses in
\fCttltrig.cf\fR should be changed to the address of any module in the
root mainframe.

.ne 4
.B Internal E143x TTLTRG driver

When you call \fIe1432_create_channel_group\fR, the last channel in
this list becomes a TTLTRG driver for the group of modules.  This
TTLTRG driver is purely an internal thing, not something the user
needs to worry about, and normally is hidden from the user.

However, you have to make sure that the TTLTRG driver ends up in the
master mainframe so it can successfully drive the TTLTRG line for all
modules in all mainframes.

For this reason, when you call \fIe1432_assign_channel_numbers\fR, you
should list the logical addresses in reverse order.  That way, the
highest channel ID (which will become the TTLTRG driver when you do
\fIe1432_create_channel_group\fR) will end up in the master mainframe.

This is opposite of what people intuitively try to do.  It means that
higher-numbered channels are in the root mainframe and lower-numbered
channels are in the non-root mainframe.  However, this setup does work
and is the one that is tested by HP.

.ne 4
.B Measurement Arm

Because of synchronization problems caused by the one-directional
nature of the TTLTRG lines, you must use manual arm, not auto arm or
RPM arm, for multi-mainframe measurements.  By manual arm, we mean
that the host computer must do the arm using \fIe1432_arm_measure\fR
(or equivalent, see ``Measurement Trigger'' below).

Although we haven't tested it, the \fIe1432_set_mmf_delay\fR function
should help work around this limitation to manual arm, if auto arm or
RPM arm really are necessary for a particular multi-mainframe
application.

.ne 4
.B Measurement Trigger

If you use manual trigger in addition to manual arm, you can simply
use \fIe1432_arm_measure\fR for the arm and
\fIe1432_trigger_measure\fR for the trigger.

However, if you want some other trigger (external trigger, input
trigger, source trigger, tach trigger), then you must use the special
multi-mainframe versions of \fIe1432_arm_measure\fR.  Instead of just
using \fIe1432_arm_measure\fR, you must call:

.ne 4
.cS
    e1432_arm_measure_master_setup(hw, master_id);
    e1432_arm_measure(hw, global_id, 0);
    e1432_arm_measure_slave_finish(hw, slave_id);
    e1432_arm_measure_master_finish(hw, master_id);
.cE

The manual pages for these function goes into a little more detail
about the IDs that should be used.

Only E143x modules in the master mainframe can trigger a
multi-mainframe E143x measurement.  This is true for any trigger mode
except manual trigger, where the trigger does not come from a module
at all.

.ne 4
.B Phase Performance

Phase specs will be degraded by the delay that the MXI cables add to
the sample clock.  A customer-written calibration to correct this is
certainly possible, but we have not tested this.  The delay may be
insignificant for many low-frequency applications, since the phase
error scales with frequency.

A system with the two MXI cards, and a 1 Meter cable, shows a 75 nsec
sample clock delay to the non-root mainframe.  This corresponds to a
-0.69 degree phase error at 25.6 kHz.

A 4 Meter cable adds about 17 nsec more delay, for a total of 92 nsec
clock delay in the non-root mainframe.  This corresponds to a -0.85
degree phase error at 25.6 kHz.

The MXI cable adds about 1.7 nsec per foot of cable.

Each daisy-chained mainframe adds another increment of delay, but only
for the additional cabling length.

In theory, it should be possible to use the SMB Trig In input on the
MXI card with an external clock source to drive all the mainframes
simultaneously.  This should eliminate most of the
mainframe-to-mainframe phase errors.  This might be useful when better
phase accuracy is desired, but the application does not want to deal
with mainframe-to-mainframe calibration and correction.

For more information about using the MXI Trig In connector, see the
end of the manual page for \fIe1432_set_clock_source\fR, and the
manual page for \fIe1432_set_auto_group_meas\fR.

.ne 4
.B Local Bus

The VXI local bus does not cross the MXI cable.  The local bus only
connects adjacent VXI modules.  It is possible to set up measurements
that use the local bus, but each mainframe will need to have its own
module that reads the E143x local bus data (such as an E1562 disk
module).

.SH "SEE ALSO"
.na
e1432_init_measure, e1432_arm_measure,
e1432_arm_measure_master_finish, e1432_set_mmf_delay,
e1432_trigger_measure
.ad
